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Abstract — We consider questions related to the computation 
of the capacity of codes that avoid forbidden difference patterns. 
The maximal number of n-bit sequences whose pairwise differ- 
ences do not contain some given forbidden difference patterns 
increases exponentially with n. The exponent is the capacity of 
the forbidden patterns, which is given by the logarithm of the 
joint spectral radius of a set of matrices constructed from the 
forbidden difference patterns. We provide a new family of bounds 
that allows for the approximation, in exponential time, of the 
capacity with arbitrary high degree of accuracy. We also provide 
a polynomial time algorithm for the problem of determining if 
the capacity of a set is positive, but we prove that the same 
problem becomes NP-hard when the sets of forbidden patterns 
are defined over an extended set of symbols. Finally, we prove 
the existence of extremal norms for the sets of matrices arising in 
the capacity computation. This result makes it possible to apply a 
specific (even though non polynomial) approximation algorithm. 
We illustrate this fact by computing exactly the capacity of codes 
that were only known approximately. 

Index Terms — Capacity of codes, joint spectral radius. 



I. Introduction 

In certain coding applications one is interested in binary 
codes whose elements avoid a set of forbidden patterns. In 
order to minimize the error probability of some particular 
magnetic -recording systems, a more complicated problem 
arises when it is desirable to find code words whose differences 
avoid forbidden patterns. 

Let {0, 1}™ denote the set of words of length n over {0, 1} 
and let u, v G {0, 1}™. The difference u — v is a word of length 
n over { — 1,0, +1} (as a shorthand we shall use {—,0,+} 
instead of { — 1,0, +1}). This difference is obtained from u 
and v by symbol-by-symbol subtraction so that, for example, 
0110 — 1011 = — h0— . Consider now a finite set D of words 
over { — , 0, +}; we think of D as a set of forbidden difference 
patterns. A set (or code) C C {0, 1}" is said to avoid the set 
D if none of the differences of words in C contain a word 
from D as subword, that is, none of the differences u — v with 
u,v £ C can be written as u — v = xdy for d £ D and some 
(possibly empty) words x and y over {— , 0, +}. 

We are interested in the largest cardinality, which we denote 
by 6 n (D), of sets of words of length n whose differences 
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avoid the forbidden patterns in I?. If the set D is empty, then 
there are no forbidden patterns and 8 n (D) — 2". When D 
is nonempty, then S n (D) grows exponentially with the word 
length n and is asymptotically equal to 2 ca P( I? )" where the 
scalar < cap(D) < 1 is the capacity of the set D. The 
capacity is thus a measure of how constraining a set D is; 
the smaller the capacity, the more constraining the forbidden 
difference patterns are. 

As an illustration consider the set of forbidden patterns 
D = {H — , ++}. Differences between two words in C = 
{U1OU2O ■ ■ ■ Ouk : Ui £ {0, 1}} will have a "0" in any succes- 
sion of two characters and will therefore not contain any of the 
forbidden patterns. From this it follows that 5 n > 2 T T ^/ 2 1 and 
so cap(£>) > 1/2. One can show in fact that cap(D) = 1/2. 
This follows from the next proposition together with the simple 
observation that the capacity of the set D = { + — ,++} is 
identical to the capacity of the set D = { + — , ++, — h, }. 

Proposition 1: The capacity of the set {+, — } m is given by 
(to — 1)/to. 

Proof: Let Ckm be a code of length km avoiding D. 
In any given window of length to, the set of words appearing 
cannot contain both u and u (we use u to denote the word 
obtained by inverting the ones and the zeros in u). This implies 
that there are at most 2 m ~ 1 different words in any given 
window of size to. Let us now consider words in Ck m as 
a concatenation of k words of length to. There are at most 
2(m-i)fe worc js in Ckm and so cap(£>) < (to — l)/m. Now 
consider the code 



C km = { Zl 0z 2 ■ ■ ■ 0z k : Zi £ {0, l}" 1 - 1 }. 



(1) 



This code satisfies the constraints, and so the bound (m— l)/m 
is reached. □ 
The computation of the capacity is not always that easy. As 
an example it is proved in [15] that the capacity of {+ + +} 
is given by log 2 ((l + {l9 + 3y%3) 1/3 + (19- 3V33) 1/3 )/3) = 
.8791 . . . and the same reference provides numerical bounds 

for the capacity of {0 H h} for which no explicit expression 

is known. 

The capacity of codes that avoid forbidden difference pat- 
terns was first introduced and studied by Moision, Orlitsky 
and Siegel. In [15], these authors provide explicit values for 
the capacity of particular sets of forbidden patterns and they 
prove that, in general, the capacity of a forbidden set D can be 
obtained as the logarithm of the joint spectral radius of a set of 
matrices with binary entries. The joint spectral radius, which 
we formally define below, is a quantity that quantifies the 
maximal asymptotic growth rate of products of matrices taken 
from a set. This quantity is notoriously difficult to compute 
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and approximate. It is known in particular that the problem 
of computing, or even approximating, the joint spectral radius 
of two matrices with binary entries is a problem that is NP- 
hard [20] and that the problem of determining if the joint 
spectral radius of two matrices with nonnegative entries is 
greater than one is undecidable [6], Moreover, the size of the 
matrices constructed in order to compute the capacity is not 
polynomial in the size of the forbidden set D : if the length 
of the forbidden words is m, the dimension of the matrices 
is 2 m ~ 1 x 2 m_1 . Hence, even constructing the matrices is 
an operation that cannot be performed in polynomial time. 
However, as pointed out in [15], the matrices that arise in the 
context of capacity computation have a particular structure and 
so these negative results do not rule out the possibility for the 
capacity to be computable in polynomial time. 

We provide several results in this paper, all are related to 
the capacity computation and its complexity. 

We first provide new bounds that relate the capacity of a set 
of forbidden patterns D with the values 5 n (D), the maximum 
size of any code of length n avoiding D. These bounds depend 
on parameters that express the number and positions of zeros 
in the patterns of D. These new bounds allow us to compute 
the capacity of any set to any given degree of accuracy 
by numerically evaluating S n (D) for increasing values of 
n. The approximation algorithm resulting from these bounds 
has exponential growth but provides an a-priori guaranteed 
precision, and so with this algorithm the computational effort 
required to compute the capacity to a given degree of accuracy 
can be evaluated before the calculations are actually started. 
As an example, it follows from the bounds we provide that the 
capacity of a set of forbidden pattern that does not contain any 
0s can be computed with an accuracy of 90% by evaluating 
5 n (D) for n = 10 (see Corollary |2]below). 

In a subsequent section, we provide explicit necessary and 
sufficient conditions for a set to have positive capacity and we 
use this condition for producing a polynomial time algorithm 
that decides whether or not the capacity of a set is positive. As 
explained above, the capacity of a set is given by the logarithm 
of the joint spectral radius of a set of matrices constructed 
from the forbidden patterns. Our polynomial time algorithm 
therefore provides a procedure for checking whether or not 
the joint spectral radius of these matrices is larger than one. 
This problem is known to be NP-hard for general matrices. 

We then consider the situation where in addition to the 
forbidden symbols — , and + the forbidden patterns in D 
may also include the symbol ±, where ± stands for both the 
symbol + and — . We prove that in this case the problem of 
computing the capacity, or even determining if this capacity 
is positive, becomes NP-hard. 

Finally, we show an algebraic property of the sets of 
matrices constructed in order to compute the capacity: there 
exists always an extremal norm for this set. This theoretical 
result makes it possible to apply specific algorithms in order 
to compute the joint spectral radius. These methods, although 
non-polynomial, can be more efficient than general ones, and 
we use them for computing the capacity of some specific 
forbidden sets. 

These results allow us to better delineate the capacity 



computation problems that are polynomial time solvable from 
those that are not. We do however not provide in this paper 
an answer to the question, which was the original motivation 
for the research reporter here, as to whether or not one 
can compute the capacity of sets of forbidden patterns over 
{—,0,+} in polynomial time. This interesting question that 
was already raised in [15], remains unsettled. 

II. Capacity and joint spectral radius 

Let D be a set of forbidden patterns over the alphabet 
{ — ,0, +} and consider for any n > 1 the largest cardinality, 
denoted by 8 n (D), of sets of words of length n whose pairwise 
differences avoid the forbidden patterns in D. The capacity of 
D is defined by 

log 2 5 n (D) 



cap(L>) 



lim 

n — >oo 



(2) 



Moision et al. show in [15] how to represent codes as 
products of matrices. Associated to any set D of forbidden 
patterns, they construct a finite set E(Z?) of matrices for which 



= max{\\A 1 ...A n \\ -.A, e E(Z>)}. 



(3) 



In this expression, the matrix norm used is the sum of 
the absolute values of the matrix entries. The matrices they 
construct are of dimension 2 m_1 x 2 m ~ 1 and have binary 
entries. For sake of conciseness, we do not reproduce here the 
explicit construction of S(-D) but refer instead the interested 
reader to [15] for more details. Combining (0 and (0 we 
deduce that 



cap(D) 



= lim 

n — *oo 

= lim 



log 2 Sjp) 
n 

log 2 6 m -l+ n (D) 

m — 1 + n 
log 2 max^iGS ||Ai . . . A r , 



= lim 

n — >oo fl 

= log, lim max II A\ . . . A n \\ 1/n 

The quantity lirrin^oo max^gs H-^-i ■ ■ ■ ^nll 1 '™ appearing 
in the last identity is a joint spectral radius. For any compact 
set of matrices A, the joint spectral radius of A is defined by 

p(A) = lim sup sup \\Ai . . .A n \\ 1/n . 

Hence we have the fundamental relation : 

cap(£>) = log 2 p(£(£>)). 

The joint spectral radius of a set of matrices is a quantity 
that was introduced by Rota and Strang [19] and that has 
received intense research attention in the last decade. For more 
references on the joint spectral radius, consult the survey [5]. 

III. Upper and lower bounds 

In this section, we derive bounds that relate the capacity of a 
set D with S n (D). Consider some set D of forbidden patterns 
and denote by r\ (respectively r 2 ) the maximal k for which 
k is the prefix (respectively suffix) of some pattern in D. No 
pattern in D begins with more than ri zeros and no pattern 
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in D ends with more than r 2 zeros. We also denote by r the 
maximal number of consecutive zeros in any pattern in D; 
obviously, r > max(ri,r2). In the next theorem we provide 
upper and lower bounds on the capacity in terms of 5 n {D). 
As n increases these bounds converge to the same identical 
limit, cap(-D), and so the bounds can be used to approximate 
the capacity to any desired degree of accuracy. 
Theorem 1: For any n > r\ + r 2 we have 

log 2 S n (D)-(r 1+ r 2 ) log^(D) 

n + r + 1 — (n + ra) n 
Proof: Let us first consider the upper bound. The fol- 
lowing equation is straightforward, given any positive integers 
k,n, and any set of forbidden patterns D: 



Skn < 52 



(5) 



Indeed, considering any word of length kn as the concatena- 
tion of k subwords of length n, for each of these subwords 
we have at most S n possibilities. Taking the ypth power of 
both sides of this inequality and taking the limit k — * oo, we 
obtain : 

2Cap(D) < gl/n 

P n < 6 n . 

Now let us consider the lower bound. The optimal code of 
length n contains at least |~2~ ri ~ r2 <5 ra (D)] words that coincide 
in the first r% bits and in the last r 2 bits (because there are in 
total 2 ri+r2 different words of length r\ + r-2). Denote the set 
of strings of all these words from (ri + l)st bit to (n — ra)th 
bit by C. This set contains at least \2-' ri ~ r2 5 n (D)] different 
words of length n — r± — r 2 ■ Then for any / > 1 the code 

C = { Ul r+1 u 2 r+1 ■ ■ • r+1 u ; r+1 , u k e C, k = 1, . . . , 1} 

avoids D. The cardinality of this code is at least 
\2- ri - r2 5 n (D)] 1 and the length of its words is N = 
l(n — r\ — r 2 + r + 1). Therefore, for any I we have 

Sn(D) > \2- r ^S n (D)Y. 

Taking the power 1/N of both sides of this inequality, we get 

' l— r2fi f£)Y\l/(n—rx—r2+r+l) 



1/N 

Sn(D) > \2- 



which as N — > oo yields 

p > \2~ ri ~ r2 5 n (D)\ l ^ n ~ ri ~ r2+r+r >. 

Now after elementary simplifications we arrive at the lower 
bound on cap(Z?). □ 

These bounds allow to design an algorithm in order to 
compute the capacity only by evaluating the successive values 
S„. For an efficient (although non-polynomial) method of 
computation of 6 n , see [13]. 

Both bounds in this theorem are sharp in the sense that 
they are both attained for particular sets D. The upper bound 
is attained for the set D = and the lower bound is attained, 
for instance, for the set D = {0 m_1 +}. Indeed, in this case 
r = r\ = m — 1, r 2 = and cap(Z?) = 0, while 5 n = 2' Tl ~ 1 
for n > m — 1. Here is a direct proof of this equality, drawn 
from [15]: Clearly, for all n > m— 1, we can construct a code 



of size S n — 2 m_1 . It happens that for any given length n 
this size is maximum. Otherwise, there must be two different 
words u and v whose prefixes of length k coincide. In order 
to avoid the forbidden pattern, the k + 1-th symbols must also 
be equal, and so on. But then both words are equal, and we 
have reached a contradiction. 

Corollary 1: Let D be given and let r, r\ and r 2 be defined 
as above. Then 

cap(£») < < I max(n + r 2 , r + 1) + cap(D). 

Proof: Simply use the fact that the capacity is always 
less than one. □ 

We may specialize these general bounds to sets of particular 
interest. 

Corollary 2: Let D be given and let r, r\ and r 2 be defined 
as above. Then 

1) If cap(£>) = the size of any code avoiding D is 
bounded above by the constant 2 Tl+r2 . 

2) If the patterns in D contain no zeros, then 

n cap(.D) < log 2 5 n (D) < (n + 1) cap(D). 

3) If none of the patterns in D start or end with a zero, 
then n cap(Z?) < log 2 5 n (D) < (n + r + 1) cap(D). 

Example 1: We do not know the capacity of the set D = 
{+0 + 0+} with a good precision, but by Corollary below, 
we know that it is positive. In this case r\ = r 2 = 0, r = 1, 
and hence, 

P n <s n <v\ 

Example 2: Consider the set D = {+ -\ }-A approxi- 
mate computation of the joint spectral radius leads to p = 
2 0.8H3... < L755j and thus> by CorollaryEl 



->0.8113n 



< S n < 1.755 2 



0.8113n 



IV. Positive capacity can be decided in polynomial 

TIME 

As previously seen by a direct argument, the capacity of 
the set {0 m_1 +} is equal to zero. In this section we provide 
a systematic way of deciding when the capacity of a set is 
equal to zero. We first provide a simple positivity criterion that 
can be verified in finite time and then exploit this criterion 
for producing a positivity checking algorithm that runs in 
polynomial time. In the sequel we shall use the notation —D 
to denote the set of elements that are the opposites to the 

elements of D, for example if D = { h0,0 } then 

-D = {+-0,0 + +}. 

Theorem 2: Let D be a set of forbidden patterns of lengths 
at most m. Then cap(£>) > if and only if there exists a word 
on the alphabet {+, — , 0} that does not contain any word of 
D U — D as sub word and that has a prefix m and a suffix 

+o m - 1 . 

Proof: If the capacity is positive, then for sufficiently 
large n there is a code avoiding D of size > 2 2m_1 + 1. 
This code has at least two words u, v with the same m-bit 
prefix and (m— l)-bit suffix (because there are in total 2 2 ™ 1-1 
different words of the length 2m — 1). Taking the difference 
u — v and removing, if necessary, several last zeros we get an 
admissible string with a prefix 0™ 1 and a suffix ±0 m_1 . If the 
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first bit in the suffix is +, then the proof is completed, if it is 
— , then we apply the same reasoning to v — u. 

Conversely, suppose there exists a feasible string d of +, — 
and of some length n. Clearly, n > 2m. Let ui,uq E 
{0, l}"-2m+x be binary words, for which m (ui-u o )0 m " 1 = 
d. Then for any I > 1 the code 

c = {« il o m - 1 -..« i ,o m - 1 , i k e{0,l},k = l,...,l} (6) 

avoids D. The cardinality of this code is 2 l . Hence 6 n (D) is 
unbounded and we conclude from Corollary [flthat the capacity 
is positive. □ 

Corollary 3: If every word in D contains at least two 
nonzero symbols, then cap(£>) > 0. 

Proof: For any such set the word d — m + m_1 is 
admissible, and by Theorem |2] the capacity is positive. □ 

Corollary 4: If D consists of one forbidden pattern p of 
length m, then its capacity is zero if and only if p has at least 
m — 1 consecutive zeros. 

Proof: If a pattern p is 0™ 1 or +0 TO_1 , then obviously 
there are no admissible strings, and by Theorem|2]the capacity 
is zero. The same holds for — m , since this is the negation 
of +0™ l ~ 1 and for m ~ 1 ± because of the symmetry. In all the 
other cases the admissible string exists and so cap(D) > 0. 
Indeed, if p has a unique nonzero bit, then the word d = 
m + +0 m ~ 1 is admissible, if it has at least two nonzero bits, 
then the proof follows from Corollary [3] □ 

We now prove the polynomial-time solvability of the 
problem of determining whether the capacity of a set D 
is positive. The proof is constructive and is based on the 
so-called Aho-Corasick automaton that checks whether a 
given text contains as a subsequence a pattern out of a given 
set [1]. Let P be a given set of patterns. The transition graph 
of the Aho-Corasick automaton for the set P is defined as 
follows : First, construct the retrieval tree, or trie, of the set 
P. The trie of P is the directed tree of which each vertex 
has a label representing a prefix of a pattern in P, and all 
prefixes are represented, including the patterns themselves. 
The root of the tree has the empty string as label. Edges have 
a label too, which is a symbol of the used alphabet. There 
is an edge labeled with the symbol a from a vertex s to a 
vertex t if t is the concatenation sa. 

In order to have an automaton, we complete the trie by 
adding edges so that for each vertex s, and each symbol 
a, there is an edge labeled a leaving s. This edge points 
to the vertex of the trie of which the label is the longest 
suffix of the concatenation sa. Note that this vertex can 
be the root (that is, the empty string) if no vertex in the 
trie is a suffix of sa. Finally, the accepting states of the 
automaton are the vertices whose labels are patterns of 
P. This automaton accepts words that contain a pattern in 
P and halts whenever this pattern is a suffix of the entered text. 

If k e D or +0* e D, k < m, then, by Theorem 
cap(D) = 0. If this is not the case, we construct the graph of 
the automaton of Aho-Corasick for the set P = D U (-D) U 
{+0" 1 " 1 }. We then remove any vertex labeled with a pattern 
in P (i.e., a state reached when a suffix of the text entered is in 
the set P) except the vertex labeled {+0 m_1 }. The size of the 



constructed graph is polynomial in the size and the number of 
the forbidden patterns. Let us now denote go m the state reached 
after entering the word m . This state is well defined since 
m doesn't contain any forbidden pattern, and hence no state 
reached after entering any prefix of the string m was removed 
from the primary automaton. We also denote q +om -i the state 
corresponding to the suffix +0 m ~ 1 for the entered text (i.e. 
the accepting state corresponding to the pattern +0 m ~ 1 in the 
Aho-Corasick automaton). We have the following criterion for 
zero-capacity: 

Theorem 3: The capacity of a set D is positive if and only 
if there is a path from go™ to q +Qm ~i in the graph constructed 
above. 

Proof: If cap(D) > 0, by Theorem |2] there exists a 
word d, beginning with m zeros, and ending with +0'™" 1 , that 
avoids D U —D. Hence, entering this word in the automaton, 
the finite state will be (well defined and will be) the vertex la- 
beled +0" l ~ 1 , because the vertices removed from the original 
automaton of Aho-Corasick do not make any problem, since 
we do not reach the vertices labeled with forbidden patterns. 
On the other hand, a path in the constructed graph represents 
an acceptable word, since it doesn't pass through any removed 
vertex, and hence no suffix of any prefix of this word will be 
in the forbidden set. 

Moreover, a shortest path will give the shortest acceptable 
word, since the length of the path is equal to the length of 
the represented word. □ 

Corollary 5: The problem of determining whether or not 
the capacity of a given set of forbidden patterns is positive 
can be solved in polynomial time. 

Proof: Aho shows in [1] that the automaton is con- 
structible in polynomial time. The determination of the state 
<?o m and the computation of the shortest path are obviously 
polynomially feasible. □ 

Corollary 6: If for a set D of forbidden patterns there are 
admissible words, then the length of a shortest admissible word 
does not exceed 2M + 2m, where m is the maximal length 
of all patterns in D and M is the sum of the lengths of each 
forbidden pattern. 

Proof: The number of vertices of the graph does not 
exceed 2M + m + 1. Indeed, for each pattern of length I in 
D U —D we add to the automaton at most I states, since there 
are no more than I prefixes of this pattern. We still add the 
pattern {+0 m_1 } (maximum m new states), and the root. If 
there is a path connecting two given vertices, this path can 
be chosen so that its length (in terms of number of vertices) 
will not exceed the total number of vertices (if it does not 
pass through the same vertex twice). Every edge of this path 
adds one bit to the admissible string. The initial length of the 
string is m (we start from 0™ 1 ), therefore the total length of 
the admissible word is at most 2M + 2m. □ 

Proposition 2: If the capacity is positive, then cap(£>) > 
1/(2M + m), where M is the total number of characters in 
D and m is their maximal length. 

Proof: If cap(D) > 0, then there is an admissible string 
of length n < 2M +2m (Corollary[6). Consider the code given 
by equation l|6). Its size is 2 l and the length of its words is at 
most 
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Ni = l(2M + 2m-m) = l(2M + m) 

Therefore 

cap(D) = lim 

" I™ l{2M+m) 2M + m ' 

□ 

V. Positive capacity is NP-hard for extended sets 

We now consider the situation where forbidden patterns are 
allowed to contain the ± symbol. The symbol ± is to be 
understood in the following sense: whenever it occurs in a 
forbidden pattern, both the occurrences of + and of — are 
forbidden at that particular location. So, for example, avoiding 
the forbidden set {0 ± +±} is equivalent to avoiding the set 
{0 + ++, + +-, - ++, - +-}. All results obtained for 
forbidden patterns over { — , 0, +} have therefore their natural 
counterparts in the situation where the forbidden patterns 
are defined over the alphabet {— ,0,+,±}. In particular, the 
results of Section 3 do transfer verbatim and the bounds 
derived in Theorem 1 are valid exactly as stated there. We 
now prove a complexity result of capacity computation in this 
set-up. 

Theorem 4: The problem of determining if the capacity of 
a set of forbidden patterns over {0, +, — , ±} is equal to zero 
is NP-hard. 

Proof: The proof proceeds by reduction from the Not- 
All-Equal 3SAT problem that is known to be NP-complete 
(see [10]). In the Not-All-Equal 3SAT problem, we are given 
m binary variables xi, . . . , x m and n clauses that each contain 
three literals (a literal can be a variable or its negation), and 
we search a truth assignment for the variables such that each 
clause has at least one true literal and one false literal. 
Suppose that we are given a set of clauses. We construct a set 
of forbidden patterns D such that cap(D) > if and only if 
the instance of Not-All-Equal 3SAT has a solution. The first 
part of D is given by: 

{(0±0),(0±±0),...,(0± m - 1 0)}. (7) 

Words over {—,0,+} that avoid these patterns are exactly 
those words for which any two consecutive zeros are either 
adjacent or have at least m symbols + or — between them. 
We use these m symbols as a way of encoding possible truth 
assignments for the variables. 

We then add to D two patterns for every clause. These 
patterns are of length m and are entirely composed of symbols 
±, except for the positions corresponding to the three variables 
of the clause, which we set to + if the clause contains the 
variable itself, or to — if the clause contains the negation of 
the variable. We also add the opposite of this pattern; this 
last pattern is not necessary for the proof but preserves the 
symmetry of the construction. 

For example, if the instance of Not- All-Equal 3SAT consists 
of the two clauses {xx.x^.x^) and (x 2 , X4, x 5 ), the corre- 
sponding set D will be D = {(0±0), (0±±0), (0±±±0), (0± 
±±±0), (+±-+±), (-±+-±), (±-±++), (±+±— )}. 



Such a set D has always a length polynomial in the number 
of clauses and the number of variables. 
We now prove that there is a solution to the instance of Not- 
All-Equal 3SAT if and only if cap(D) > 0. First, suppose that 
there exists a satisfying truth assignment for x and denote it 
by . . . ,uj m }. Associated to any k > 1 we construct a 
code of length k(m + 1) containing 2 k words as follows: 

Cfe(m+i) = {OwOwOwO • • • OwOu, OujOljOujO ■ ■ ■ OcjOw, 

OujOujOluO ■ ■ ■ OluOlu, OwOwOwO • • • OwOw}, 

where w = u)\ ■ ■ -uj m - 

Any difference between two words in this code is a word 
of the form OZ1OZ2O • • • Ozk where for every 1 < i < k, 
Zi is either a sequence of m 0's or a word of length m 
over { — ,+}. Because ui satisfies the instance of Not-All- 
Equal 3SAT, these words avoid the set D constructed above. 
Moreover, the cardinality of C fc ( m+1 ) is 2 fc and hence 

cap(L>) > lim log 2 = > 0. (8) 

fc^oo m + 1 

For the reverse implication, assume now that cap(D) > 
0. The capacity is positive, and so one can find two words 
whose differences contain a and a +. But then since this 
difference must avoid the first part of the forbidden pattern, 
for a code C large enough, there must exist two words in the 
code whose difference contains a word over { — , +} of length 
m. But this sequence avoids also the second part of D, and 
thus it represents an acceptable solution to our instance of 
Not-All-Equal 3SAT. □ 

Note that a similar proof can be given if we replace the 
symbol "±" in the statement of the theorem by a symbol that 
represents either +, — , or 0. 

VI. Extremal norms and computing of the 

CAPACITY 

A classical way to estimate a joint spectral radius consists 
of computing successive upper bounds on it by applying the 
following well known inequality 

p n (S)<max{||i4i...A n || : A, e S} 

that holds for any norm, and any length n of the products. One 
could then hope that for a well-chosen norm, the joint spectral 
radius would be already obtained for n = 1, that is, for the set 
of matrices itself. This is the concept of extremal norm, which 
we now define properly : A norm || • || in M. d is called extremal 
for a family of operators A\, . . . , A r if \\Ai\\ < p(A\, . . . , A r ) 
for all i = 1, . . . , r. The unit ball M of this norm is called 
the extremal convex body. 

The notion extremal is justified by the fact that p is the 
smallest possible value such that the norms of all the operators 
A\, . . . , A r do not exceed this value. The above inequality, 
which holds for any norm, for extremal norms becomes an 
equality (for all n > 1). If M is the unit ball corresponding 
to an extremal norm (an extremal body), AiM C pM. On 
the other hand, any convex body M (convex compact with 
nonempty interior centrally symmetric with respect to the 
origin) possessing this property generates an extremal norm. 
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It suffices to take the Minkowski norm defined by this body: 
||x|| = inf{A > 0, jx £ M}. Thus, there is a natural one- 
to-one correspondence between extremal norms and extremal 
bodies. 

The existence of an extremal norm can simplify many prob- 
lems related to the joint spectral radius, see [2], [16] and [18] 
for details. However, not every set of matrices possesses an 
extremal norm. The corresponding counterexamples are simple 
and well-known. Sufficient conditions for the existence of an 
extremal norm can be found in [2], [16], [17]. For the matrices 
arising in the context of the capacity computation, however, 
these conditions are not always satisfied. Nevertheless, it turns 
out that in the case of capacity computation, the matrices do 
in fact always possess an extremal norm. 

Theorem 5: For any set D of forbidden patterns the set 
X(-D) possesses an extremal norm. 

Proof: Let £(Z?) be the set of matrices corresponding 
to D. For a given point x > let 

0(x) = {p- n Ai...AnX, AiGE(D) n > oj 

be the normalized orbit of the point x under the action of 
all possible products of the operators in S. The product of 
length zero is defined as the identity operator, so the set 0(x) 
contains x. Now define a set M as follows 



After N = 



3^ In 23- 



steps the algorithm terminates. The 



,1/(^+1) 



M = Conv{o(ej), 



j = l, 



where ei, . . . , e 2m -i are the canonical basis vectors in R 2 , 
and Conv denotes the convex hull. The set M is obviously 
convex, centrally symmetric with respect to the origin, and 
possesses a nonempty interior (because it contains the cross- 
polytope with the vertices {±ej, j = 1, ...,2 m_1 } whose 
interior is nonempty). Moreover, Theorem ^ implies that the 
set 0(x) is bounded for any x, therefore M is bounded. So M 
is a convex body that possesses the property AiM C pM, i = 
1, ...,r. Therefore it generates an extremal norm, and the 
theorem follows. 

□ 

The very existence of an extremal norm for a set of matrices 
makes it possible to apply a geometric algorithm for comput- 
ing a joint spectral radius with a given relative precision e. We 
now briefly describe this algorithm; for all technical details we 
refer the reader to [16]. For the sake of simplicity we consider 
the case of two matrices, the case of an arbitrary number of 
matrices is treated in the same way. 

The algorithm. Suppose operators Aq,A\ acting in R d 
possess an extremal norm; one needs to find a number p* 

such that P l < e, where e > is a given accuracy. 
Consider a sequence of convex polytopes {P k } produced as 
follows. Pq = I (2:1, . . . ,Xd) € R d , X) \ x i\ < l| is a cross- 
poly tope. For any k > the poly tope Pk+i is an arbitrary 
polytope possessing the following properties: it is symmetric 
with respect to the origin, has at most q(e) = CdS^~ 
vertices, where is an effective constant depending only 
on d, and (1 - e)AP k C P k+ i C AP k , where AX = 
Conv{A X,AiX}. 



value p* = (vn+i)"' y± ~ ' gives the desirable approximation 
of the joint spectral radius. Here v k is the biggest distance from 
the origin to the vertices of the polytope P k , Ci, c 2 are lower 
and upper bounds of the values {p~ -pfc(£, || -||), k E N}. For 
the adjacency matrices the values c\, c 2 can be taken directly 
from Theorem ^ 

Each step requires us taking the convex hull of two poly- 
topes having at most q(e) vertices and requires the approxi- 
mation of one polytope with 2q(e) vertices by a polytope with 
q(e) vertices with accuracy e. Both operations are known to 
be polynomial w.r.t. ~ [16] (the dimension d is fixed). The 
computational complexity of this algorithm is C-e ~ , where 
C is some constant and d = 2 m ~ 1 . Therefore the algorithm 
is applicable for small values of m, say, for m < 6. 

The complexity of this algorithm is exponential with respect 
to m, as the one proposed in Section |lll| that approximates the 
capacity by successive estimations of 8 n . The advantages of 
one algorithm over the other appears in numerical computation 
of the capacity. In many cases the approximation of invariant 
bodies by polytopes can lead to the exact value of the 
joint spectral radius. Suppose that by numerical observations 
we conjecture that p is attained by some product n„ = 
Ai 1 ...Ai n , i.e. p = /^ILj) 1 /". If during the calculations 
we find a polytope P such that AP C p(U n ) 1/n P, then 
it occurs that p = p(II n ) 1 / rt . As the polytope P we take 
P = Pfc = Convl/Pu , A^v , j — 0, . . . , k} for some integer 
k, where v is the eigenvector of n„ corresponding to the 
largest by modulo eigenvalue (we assume that this is real and 
unique). 

Let us illustrate this method by computing the exact values 
of the capacity for several codes. In Examples [3] and |4] we 
find the values of capacities that were approximated in [15]. 
Example |3 deals with a code with m = 4. 

Example 3: cap({0 + +}) = log 2 p(A ) = log 2 (^|±±) = 
0.69424191 .... The eigenvector is v = (2, \/5 - 1, 2, y/% - 
1) T . The algorithm terminates after five steps, the polytope 
P = P5 has 32 vertices. 

Example 4: cap({0 + -}) = log 2( oL4 ) = log 2 (^|±i). 
The algorithm terminates after four steps, v — (2, y/E—1, V5— 
1,2) T , P = P4, the polytope has 40 vertices. 



Example 5: cap({+ + +-}) = log 2 ( ^ 3+ | Vg ±^) = 
log 2 y /p(A A 1 ) = 0.90053676 .... The algorithm terminates 
after eleven steps, the polytope P = P\x has 528 vertices. 

As illustrated in many applications it is quite often the case 
that the joint spectral radius is attained by some finite product. 
We say in these cases that the set of matrices possess the 
finiteness property. It was conjectured that all sets of matrices 
have the finiteness property : this is the well known finiteness 
conjecture which has been disproved in [8], [7], and [12]. 
Nevertheless, we conjecture here that the sets of matrices 
constructed in order to compute a capacity do always possess 
the finiteness property. Numerical results in [15], [11], and in 
this paper seem to support this conjecture. 
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VII. Conclusion 

One way to compute the capacity of a set of forbidden 
patterns is to compute the joint spectral radius of a set of 
matrices. In practice, this leads to a double difficulty : first, 
the size of the matrices is exponential in the size of the set 
of forbidden patterns, and second, the joint spectral radius 
is in general NP-hard to compute. Actually, it is even NP- 
hard to decide whether a joint spectral radius of two matrices 
is greater than one. We show in this paper that the simpler 
problem of checking the positivity of the capacity of a set 
defined on {+,—,0} is polynomially decidable but that the 
same problem becomes NP-hard when defined over the al- 
phabet {+, — , 0, ±}. We also provide bounds that allow faster 
computation of the capacity. Finally we prove the existence 
of extremal norms for the sets of matrices arising in the 
capacity computation and present a geometrical algorithm 
for capacity computation which we illustrate with several 
numerical examples. Even if this latter result allows to use 
algorithms that have proved to be quite efficient in practice, 
we should keep in mind that the approach that consists of 
computing the joint spectral radius of the matrices defined in 
[15] cannot lead to a polynomial algorithm to compute the 
capacity because of the exponential size of the matrices. 
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